Measuring Health Care Work-Related Contextual Factors: Development of the McGill Context Tool.

INTRODUCTION: Contextual factors can influence healthcare professionals' (HCPs) competencies, yet there is a scarcity of research on how to optimally measure these factors. The aim of this study was to develop and validate a comprehensive tool for HCPs to document the contextual factors likely to influence the maintenance, development, and deployment of professional competencies. METHODS: We used DeVellis' 8-step process for scale development and Messick's unified theory of validity to inform the development and validation of the context tool. Building on results from a scoping review, we generated an item pool of contextual factors articulated around five themes: Leadership and Agency, Values, Policies, Supports, and Demands. A first version of the tool was pilot tested with 127 HCPs and analyzed using the classical test theory. A second version was tested on a larger sample (n = 581) and analyzed using the Rasch rating scale model. RESULTS: First version of the tool: we piloted 117 items that were grouped as per the themes related to contextual factors and rated on a 5-point Likert scale. Cronbach alpha for the set of 12 retained items per scale ranged from 0.75 to 0.94. Second version of the tool included 60 items: Rasch analysis showed that four of the five scales (ie, Leadership and Agency, Values, Policies, Supports) can be used as unidimensional scales, whereas the fifth scale (Demands) had to be split into two unidimensional scales (Demands and Overdemands). DISCUSSION: Validity evidence documented for content and internal structure is encouraging and supports the use of the McGill context tool. Future research will provide additional validity evidence and cross-cultural translation.

The Definitions and Conceptualizations of the Practice Context in the Health Professions: A Scoping Review.

INTRODUCTION: Health care professionals work in different contexts, which can influence professional competencies. Despite existing literature on the impact of context on practice, the nature and influence of contextual characteristics, and how context is defined and measured, remain poorly understood. The aim of this study was to map the breadth and depth of the literature on how context is defined and measured and the contextual characteristics that may influence professional competencies. METHODS: A scoping review using Arksey and O'Malley's framework. We searched MEDLINE (Ovid) and CINAHL (EBSCO). Our inclusion criteria were studies that reported on context or relationships between contextual characteristics and professional competencies or that measured context. We extracted data on context definitions, context measures and their psychometric properties, and contextual characteristics influencing professional competencies. We performed numerical and qualitative analyses. RESULTS: After duplicate removal, 9106 citations were screened and 283 were retained. We compiled a list of 67 context definitions and 112 available measures, with or without psychometric properties. We identified 60 contextual factors and organized them into five themes: Leadership and Agency, Values, Policies, Supports, and Demands. DISCUSSION: Context is a complex construct that covers a wide array of dimensions. Measures are available, but none include the five dimensions in one single measure or focus on items targeting the likelihood of context influencing several competencies. Given that the practice context plays a critical role in health care professionals' competencies, stakeholders from all sectors (education, practice, and policy) should work together to address those contextual characteristics that can adversely influence practice.

Are different station formats assessing different dimensions in multiple mini-interviews? Findings from the Canadian integrated French multiple mini-interviews.

BACKGROUND: Multiple mini-interviews (MMI) are used to assess non-academic attributes for selection in medicine and other healthcare professions. It remains unclear if different MMI station formats (discussions, role-plays, collaboration) assess different dimensions. METHODS: Based on station formats of the 2018 and 2019 Integrated French MMI (IFMMI), which comprised five discussions, three role-plays and two collaboration stations, the authors performed confirmatory factor analysis (CFA) using the lavaan 0.6-5 R package and compared a one-factor solution to a three-factor solution for scores of the 2018 (n = 1438) and 2019 (n = 1440) cohorts of the IFMMI across three medical schools in Quebec, Canada. RESULTS: The three-factor solution was retained, with discussions, role-plays and collaboration stations all loading adequately with their scores. Furthermore, all three factors had moderate-to-high covariance (range 0.44 to 0.64). The model fit was also excellent with a Comparative fit index (CFI) of 0.983 (good if > 0.9), a Tucker Lewis index of 0.976 (good if > 0.95), a Standardized Root Mean Square Residual of 0.021 (good if < .08) and a Root Mean Square Error of 0.023 (good if < 0.08) for 2018 and similar results for 2019. In comparison, the single factor solution presented a lower fit (CFI = 0.819, TLI = 0.767, SRMR = 0.049 and RMSEA = 0.070). CONCLUSIONS: The IFMMI assessed three dimensions that were related to stations formats, a finding that was consistent across two cohorts. This suggests that different station formats may be assessing different skills, and has implications for the choice of appropriate reliability metrics and the interpretation of scores. Further studies should try to characterize the underlying constructs associated with each station format and look for differential predictive validity according to these formats.

Effect of station format on the psychometric properties of Multiple Mini Interviews.

BACKGROUND: Given the widespread use of Multiple Mini Interviews (MMIs), their impact on the selection of candidates and the considerable resources invested in preparing and administering them, it is essential to ensure their quality. Given the variety of station formats used and the degree to which that factor resides in the control of training programmes that we know so little about, format's effect on MMI quality is a considerable oversight. This study assessed the effect of two popular station formats (interview vs. role-play) on the psychometric properties of MMIs. METHODS: We analysed candidate data from the first 8 years of the Integrated French MMIs (IF-MMI) (2010-2017, n = 11 761 applicants), an MMI organised yearly by three francophone universities and administered at four testing sites located in two Canadian provinces. There were 84 role-play and 96 interview stations administered, totalling 180 stations. Mixed design analyses of variance (ANOVAs) were used to test the effect of station format on candidates' scores and stations' discrimination. Cronbach's alpha coefficients for interview and role-play stations were also compared. Predictive validity of both station formats was estimated with a mixed multiple linear regression model testing the relation between interview and role-play scores with average clerkship performance for those who gained entry to medical school (n = 462). RESULTS: Role-play stations (M = 20.67, standard deviation [SD] = 3.38) had a slightly lower mean score than interview stations (M = 21.36, SD = 3.08), p < 0.01, Cohen's d = 0.2. The correlation between role-play and interview stations scores was r = 0.5 (p < 0.01). Discrimination coefficients, Cronbach's alpha and predictive validity statistics did not vary by station format. CONCLUSION: Interview and role-play stations have comparable psychometric properties, suggesting format to be interchangeable. Programmes should select station format based on match to the personal qualities for which they are trying to select.

[Feedback Guide for direct observation of family medicine residents in Canada: a francophone tool]. / Développement et mise à l'essai du Guide de rétroaction francophone pour l'observation directe des résidents en médecine familiale au Canada.

Background: There is no CanMEDS-FM-based milestone tool to guide feedback during direct observation (DO). We have developed a guide to support documentation of feedback for direct observation (DO) in Canadian family medicine (FM) programs. Methods: The Guide was designed in three phases with the collaboration of five Canadian FM programs with at least a French-speaking teaching site: 1) literature review and needs assessment; 2) development of the DO Feedback Guide; 3) testing the Guide in a video simulation context with qualitative content analysis. Results: Phase 1 demonstrated the need for a narrative guide aimed at 1) specifying mutual expectations according to the resident's level of training and the clinical context, 2) providing the supervisor with tools and structure in his observations 3) to facilitate documentation of feedback. Phase 2 made it possible to develop the Guide, in paper and electronic formats, meeting the needs identified. In phase 3, 15 supervisors used the guide for three levels of residence. The Guide was adjusted following this testing to recall the phases of the clinical encounter that were often forgotten during feedback (before consultation, diagnosis and follow-up), and to suggest types of formulation to be favored (stimulating questions, questions of clarification, reflections). Conclusion: Based on evidence and a collaborative approach, this Guide will equip French-speaking Canadian supervisors and residents performing DO in family medicine.

Psychometric validation of the Laval developmental benchmarks scale for family medicine.

BACKGROUND: With the implementation of competency-based education in family medicine, there is a need for summative end-of-rotation assessments that are criterion-referenced rather than normative. Laval University's family residency program therefore developed the Laval Developmental Benchmarks Scale for Family Medicine (DBS-FM), based on competency milestones. This psychometric validation study investigates its internal structure and its relation to another variable, two sources of validity evidence. METHODS: We used assessment data from a cohort of residents (n = 1432 assessments) and the Rasch Rating Scale Model to investigate its reliability, dimensionality, rating scale functioning, targeting of items to residents' competency levels, biases (differential item functioning), items hierarchy (adequacy of milestones ordering), and score responsiveness. Convergent validity was estimated by its correlation with the clinical rotation decision (pass, in difficulty/fail). RESULTS: The DBS-FM can be considered as a unidimensional scale with good reliability for non-extreme scores (.83). The correlation between expected and empirical items hierarchies was of .78, p < .0001.Year 2 residents achieved higher scores than year 1 residents. It was associated with the clinical rotation decision. CONCLUSION: Advancing its validation, this study found that the DBS-FM has a sound internal structure and demonstrates convergent validity.

An 11-Item Measure of User- and Human-Centered Design for Personal Health Tools (UCD-11): Development and Validation.

BACKGROUND: Researchers developing personal health tools employ a range of approaches to involve prospective users in design and development. OBJECTIVE: The aim of this paper was to develop a validated measure of the human- or user-centeredness of design and development processes for personal health tools. METHODS: We conducted a psychometric analysis of data from a previous systematic review of the design and development processes of 348 personal health tools. Using a conceptual framework of user-centered design, our team of patients, caregivers, health professionals, tool developers, and researchers analyzed how specific practices in tool design and development might be combined and used as a measure. We prioritized variables according to their importance within the conceptual framework and validated the resultant measure using principal component analysis with Varimax rotation, classical item analysis, and confirmatory factor analysis. RESULTS: We retained 11 items in a 3-factor structure explaining 68% of the variance in the data. The Cronbach alpha was .72. Confirmatory factor analysis supported our hypothesis of a latent construct of user-centeredness. Items were whether or not: (1) patient, family, caregiver, or surrogate users were involved in the steps that help tool developers understand users or (2) develop a prototype, (3) asked their opinions, (4) observed using the tool or (5) involved in steps intended to evaluate the tool, (6) the process had 3 or more iterative cycles, (7) changes between cycles were explicitly reported, (8) health professionals were asked their opinion and (9) consulted before the first prototype was developed or (10) between initial and final prototypes, and (11) a panel of other experts was involved. CONCLUSIONS: The User-Centered Design 11-item measure (UCD-11) may be used to quantitatively document the user/human-centeredness of design and development processes of patient-centered tools. By building an evidence base about such processes, we can help ensure that tools are adapted to people who will use them, rather than requiring people to adapt to tools.

User Involvement in the Design and Development of Patient Decision Aids and Other Personal Health Tools: A Systematic Review.

BACKGROUND: When designing and developing patient decision aids, guidelines recommend involving patients and stakeholders. There are myriad ways to do this. We aimed to describe how such involvement occurs by synthesizing reports of patient decision aid design and development within a user-centered design framework and to provide context by synthesizing reports of user-centered design applied to other personal health tools. METHODS: We included articles describing at least one development step of 1) a patient decision aid, 2) user- or human-centered design of another personal health tool, or 3) evaluation of these. We organized data within a user-centered design framework comprising 3 elements in iterative cycles: understanding users, developing/refining prototype, and observing users. RESULTS: We included 607 articles describing 325 patient decision aid projects and 65 other personal health tool projects. Fifty percent of patient decision aid projects reported involving users in at least 1 step for understanding users, 35% in at least 1 step for developing/refining the prototype, and 84% in at least 1 step for observing users' interaction with the prototype. In comparison, other personal health tool projects reported 91%, 49%, and 92%, respectively. A total of 74% of patient decision aid projects and 92% of other personal health tool projects reported iterative processes, both with a median of 3 iterative cycles. Preliminary evaluations such as usability or feasibility testing were reported in 66% of patient decision aid projects and 89% of other personal health tool projects. CONCLUSIONS: By synthesizing design and development practices, we offer evidence-based portraits of user involvement. Those wishing to further align patient decision aid design and development with user-centered design methods could involve users earlier, design and develop iteratively, and report processes in greater detail.

Sound Practices: An Exploratory Study of Building and Monitoring Multiple-Choice Exams at Canadian Undergraduate Medical Education Programs.

PURPOSE: Written examinations such as multiple-choice question (MCQ) exams are a key assessment strategy in health professions education (HPE), frequently used to provide feedback, to determine competency, or for licensure decisions. However, traditional psychometric approaches for monitoring the quality of written exams, defined as items that are discriminant and contribute to increase the overall reliability and validity of the exam scores, usually warrant larger samples than are typically available in HPE contexts. The authors conducted a descriptive exploratory study to document how undergraduate medical education (UME) programs ensure the quality of their written exams, particularly MCQs. METHOD: Using a qualitative descriptive methodology, the authors conducted semistructured interviews with 16 key informants from 10 Canadian UME programs in 2018. Interviews were transcribed, anonymized, coded by the primary investigator, and co-coded by a second team member. Data collection and analysis were conducted iteratively. Research team members engaged in analysis across phases, and consensus was reached on the interpretation of findings via group discussion. RESULTS: Participants focused their answers around MCQ-related practices, reporting using several indicators of quality such as alignment between items and course objectives and psychometric properties (difficulty and discrimination). The authors clustered findings around 5 main themes: processes for creating MCQ exams, processes for building quality MCQ exams, processes for monitoring the quality of MCQ exams, motivation to build quality MCQ exams, and suggestions for improving processes. CONCLUSIONS: Participants reported engaging multiple strategies to ensure the quality of MCQ exams. Assessment quality considerations were integrated throughout the development and validation phases, reflecting recent work regarding validity as a social imperative.

Accuracy of rating scale interval values used in multiple mini-interviews: a mixed methods study.

When determining the score given to candidates in multiple mini-interview (MMI) stations, raters have to translate a narrative judgment to an ordinal rating scale. When adding individual scores to calculate final ranking, it is generally presumed that the values of possible scores on the evaluation grid are separated by constant intervals, following a linear function, although this assumption is seldom validated with raters themselves. Inaccurate interval values could lead to systemic bias that could potentially distort candidates' final cumulative scores. The aim of this study was to establish rating scale values based on rater's intent, to validate these with an independent quantitative method, to explore their impact on final score, and to appraise their meaning according to experienced MMI interviewers. A 4-round consensus-group exercise was independently conducted with 42 MMI interviewers who were asked to determine relative values for the 6-point rating scale (from A to F) used in the Canadian integrated French MMI (IFMMI). In parallel, relative values were also calculated for each option of the scale by comparing the average scores concurrently given to the same individual in other stations every time that option was selected during three consecutive IFMMI years. Data from the same three cohorts was used to simulate the impact of using new score values on final rankings. Comments from the consensus group exercise were reviewed independently by two authors to explore raters' rationale for choosing specific values. Relative to the maximum (A = 100%) and minimum (F = 0%), experienced raters concluded to values of 86.7% (95% CI 86.3-87.1), 69.5% (68.9-70.1), 51.2% (50.6-51.8), and 29.3% (28.1-30.5), for scores of B, C, D and E respectively. The concurrent score approach was based on 43,412 IFMMI stations performed by 4345 medical school applicants. It provided quasi-identical values of 87.1% (82.4-91.5), 70.4% (66.1-74.7), 51.2% (47.1-55.3) and 31.8% (27.9-35.7), respectively. Qualitative analysis explained that while high scores are usually based on minor details of relatively low importance, low scores are usually attributed for more serious offenses and were assumed by the raters to carry more weight in the final score. Individual drop or increase in final MMI ranking with the use of new scale values ranged from - 21 to + 5 percentiles, with the average candidate changing by ± 1.4 percentiles. Consulting with experienced interviewers is a simple and effective approach to establish rating scale values that truly reflects raters' intent in MMI, thus improving the accuracy of the instrument and contributing to the general fairness of the process.

Development and validation of five behavioral indices of flood adaptation.

BACKGROUND: In the current context of climate change, climate forecasts for the province of Quebec (Canada) are a lengthening of the thunderstorm season and an increase in episodes of intense precipitations. These changes in the distribution of precipitations could heighten the intensity or frequency of floods, a natural hazard that concerns 80% of Quebec's riverside municipalities. For the health and safety of the at-risk population, it is very important to make sure they have acquired necessary adaptive behaviors against flooding hazard. However, there has been no assessment of these flood adaptation behaviors to date. Thus, the aim of this study was to develop and validate five indices of adaptation to flooding. METHODS: A sample of 1951 adults completed a questionnaire by phone. The questionnaire, specifically developed for this study, measured whether they did or did not adopt the behaviors that are proposed by public health officials to protect themselves against flooding. RESULTS: The results of the item, confirmatory factor, and multiple correspondence analyses contributed to the development of five indices corresponding to the adaptation behaviors to adopt according to the chronology of events: (a) pre-alert preventive behaviors, (b) behaviors to carry out after the alert is issued, (c) behaviors to adopt during a flood not requiring evacuation, (d) behaviors to adopt during a flood requiring evacuation, and (e) post-flood behaviors. The results of this study also showed that people who perceive a risk of flooding in their home in the next 5 years tend to adopt more preventive behaviors and adaptation behaviors than those who perceive little or no risk at all. They also reveal that people who feel more adverse effects on their physical or mental health tend to adopt more adaptive behaviors than those who feel little or no adverse effects on their health. CONCLUSION: Across a series of psychometric analyses, the results showed that these flood adaptation indices could properly measure a vast range of adaptive behaviors according to the chronology of events. Therefore, researchers, public health agencies, and professionals can use them to monitor the evolution of individuals' adaptive behaviors during floods.

Impact of a Student-Driven, Virtual Patient Application on Objective Structured Clinical Examination Performance: Observational Study.

BACKGROUND: Peer-assisted learning (PAL) refers to a learning activity whereby students of similar academic level teach and learn from one another. Groupe de perfectionnement des habiletés cliniques (Clinical Skills Improvement Group), a student organization at Université Laval, Canada, propelled PAL into the digital era by creating a collaborative virtual patient platform. Medical interviews can be completed in pairs (a student-patient and a student-doctor) through an interactive Web-based application, which generates a score (weighted for key questions) and automated feedback. OBJECTIVES: The aim of the study was to measure the pedagogical impact of the application on the score at medical interview stations at the summative preclerkship Objective Structured Clinical Examination (OSCE). METHODS: We measured the use of the application (cases completed, mean score) in the 2 months preceding the OSCE. We also accessed the results of medical interview stations at the preclerkship summative OSCE. We analyzed whether using the application was associated with higher scores and/or better passing grades (≥60%) at the OSCE. Finally, we produced an online form where students could comment on their appreciation of the application. RESULTS: Of the 206 students completing the preclerkship summative OSCE, 170 (82.5%) were registered users on the application, completing a total of 3133 cases (18 by active user in average, 7 minutes by case in average). The appreciation questionnaire was answered online by 45 students who mentioned appreciating the intuitive, easy-to-use, and interactive design, the diversity of cases, and the automated feedback. Using the application was associated with reduced reported stress, improved scores (P=.04), and improved passing rates (P=.11) at the preclerkship summative OSCE. CONCLUSIONS: This study suggests that PAL can go far beyond small-group teaching, showing students' potential to create helpful pedagogical tools for their peers.

Detecting rater bias using a person-fit statistic: a Monte Carlo simulation study.

INTRODUCTION: With the Standards voicing concern for the appropriateness of response processes, we need to explore strategies that would allow us to identify inappropriate rater response processes. Although certain statistics can be used to help detect rater bias, their use is complicated by either a lack of data about their actual power to detect rater bias or the difficulty related to their application in the context of health professions education. This exploratory study aimed to establish the worthiness of pursuing the use of l z to detect rater bias. METHODS: We conducted a Monte Carlo simulation study to investigate the power of a specific detection statistic, that is: the standardized likelihood l z person-fit statistics (PFS). Our primary outcome was the detection rate of biased raters, namely: raters whom we manipulated into being either stringent (giving lower scores) or lenient (giving higher scores), using the l z statistic while controlling for the number of biased raters in a sample (6 levels) and the rate of bias per rater (6 levels). RESULTS: Overall, stringent raters (M = 0.84, SD = 0.23) were easier to detect than lenient raters (M = 0.31, SD = 0.28). More biased raters were easier to detect then less biased raters (60% bias: 62, SD = 0.37; 10% bias: 43, SD = 0.36). DISCUSSION: The PFS l z seems to offer an interesting potential to identify biased raters. We observed detection rates as high as 90% for stringent raters, for whom we manipulated more than half their checklist. Although we observed very interesting results, we cannot generalize these results to the use of PFS with estimated item/station parameters or real data. Such studies should be conducted to assess the feasibility of using PFS to identify rater bias.

Twelve Lessons Learned for Effective Research Partnerships Between Patients, Caregivers, Clinicians, Academic Researchers, and Other Stakeholders.

Research increasingly means that patients, caregivers, health professionals, other stakeholders, and academic investigators work in partnership. This requires effective collaboration rooted in mutual respect, involvement of all participants, and good communication. Having conducted such partnered research over multiple projects, and having recently completed a project together funded by the Patient-Centered Outcomes Research Institute, we collaboratively developed a list of 12 lessons we have learned about how to ensure effective research partnerships. To foster a culture of mutual respect, hold early in-person meetings, with introductions focused on motivation, offer appropriate orientation for everyone, and maintain awareness of individual and project goals. To actively involve all team members, it is important to ensure sufficient funding for everyone's participation, to ask for and recognize diverse contributions, and to seek the input of quiet members. To facilitate good communication, teams should carefully consider labels, avoid jargon and acronyms, judiciously use homogeneous and heterogeneous subgroups, and keep progress visible. In offering pragmatic, actionable lessons we have learned through our separate and shared experiences, we hope to help foster more patient-centered research via productive and enjoyable research collaborations.

Validation of the 5-item doctor-patient communication competency instrument for medical students (DPCC-MS) using two years of assessment data.

BACKGROUND: Medical students on clinical rotations have to be assessed on several competencies at the end of each clinical rotation, pointing to the need for short, reliable, and valid assessment instruments of each competency. Doctor patient communication is a central competency targeted by medical schools however, there are no published short (i.e. less than 10 items), reliable and valid instruments to assess doctor-patient communication competency. The Faculty of Medicine of Laval University recently developed a 5-item Doctor-Patient Communication Competency instrument for Medical Students (DPCC-MS), based on the Patient Centered Clinical Method conceptual framework, which provides a global summative end-of-rotation assessment of doctor-patient communication. We conducted a psychometric validation of this instrument and present validity evidence based on the response process, internal structure and relation to other variables using two years of assessment data. METHODS: We conducted the study in two phases. In phase 1, we drew on 4991 student DPCC-MS assessments (two years). We conducted descriptive statistics, a confirmatory factor analysis (CFA), and tested the correlation between the DPCC-MS and the Multiple Mini Interviews (MMI) scores. In phase 2, eleven clinical teachers assessed the performance of 35 medical students in an objective structured clinical examination station using the DPCC-MS, a 15-item instrument developed by Côté et al. (published in 2001), and a 2-item global assessment. We compared the DPCC-MS to the longer Côté et al. instrument based on internal consistency, coefficient of variation, convergent validity, and inter-rater reliability. RESULTS: Phase 1: Cronbach's alpha was acceptable (.75 and .83). Inter-item correlations were positive and the discrimination index was above .30 for all items. CFA supported a unidimensional structure. DPCC-MS and MMI scores were correlated. Phase 2: The DPCC-MS and the Côté et al. instrument had similar internal consistency and convergent validity, but the DPCC-MS had better inter-rater reliability (mean ICC = .61). CONCLUSIONS: The DPCC-MS provides an internally consistent and valid assessment of medical students' communication with patients.

Development and Validation of a Behavioural Index for Adaptation to High Summer Temperatures among Urban Dwellers.

One of the consequences of climate change is the growing number of extreme weather events, including heat waves, which have substantial impacts on the health of populations. From a public health standpoint, it is vital to ensure that people can adapt to high heat, particularly in cities where heat islands abound. Identifying indicators to include in a parsimonious index would help better differentiate individuals who adapt well to heat from those who do not adapt as well. This study aimed at developing and validating a summer heat adaptation index for residents of the 10 largest cities in the province of Québec, Canada. A sample of 2000 adults in 2015 and 1030 adults in 2016 completed a telephone questionnaire addressing their adoption (or non-adoption) of behaviours recommended by public health agencies to protect themselves during periods of high temperature, and their perceptions of how high summer heat affects their mental and physical health. Item analysis, confirmatory factor analysis, multiple correspondence analysis, measurement invariance analyses and criterion-validity analyses were used to develop a 12-behaviour heat adaptation index for distinguishing between individuals who adapt well to high temperatures and those who do not adapt as well. The results indicated that the measurement and the factor structure of the index were invariant (equivalent) across the two independent samples of participants who completed the questionnaire at different times one year apart, an important prerequisite for unambiguous interpretation of index scores across groups and over time. The results also showed that individuals who perceived more adverse effects on their physical or mental health adopted more preventive behaviours during periods of high temperatures and humidity conditions compared to those who felt lesser or no effects. This study thus presents support for the validity of the index that could be used in future studies to monitor preventive behaviours adoption during summer periods of high temperature.

Involving members of vulnerable populations in the development of patient decision aids: a mixed methods sequential explanatory study.

BACKGROUND: Patient decision aids aim to present evidence relevant to a health decision in understandable ways to support patients through the process of making evidence-informed, values-congruent health decisions. It is recommended that, when developing these tools, teams involve people who may ultimately use them. However, there is little empirical evidence about how best to undertake this involvement, particularly for specific populations of users such as vulnerable populations. METHODS: To describe and compare the development practices of research teams that did and did not specifically involve members of vulnerable populations in the development of patient decision aids, we conducted a secondary analysis of data from a systematic review about the development processes of patient decision aids. Then, to further explain our quantitative results, we conducted semi-structured telephone interviews with 10 teams: 6 that had specifically involved members of vulnerable populations and 4 that had not. Two independent analysts thematically coded transcribed interviews. RESULTS: Out of a total of 187 decision aid development projects, 30 (16%) specifically involved members of vulnerable populations. The specific involvement of members of vulnerable populations in the development process was associated with conducting informal needs assessment activities (73% vs. 40%, OR 2.96, 95% CI 1.18-7.99, P = .02) and recruiting participants through community-based organizations (40% vs. 11%, OR 3.48, 95% CI 1.23-9.83, P = .02). In interviews, all developers highlighted the importance, value and challenges of involving potential users. Interviews with developers whose projects had involved members of vulnerable populations suggested that informal needs assessment activities served to center the decision aid around users' needs, to better avoid stigma, and to ensure that the topic truly matters to the community. Partnering with community-based organizations may facilitate relationships of trust and may also provide a non-threatening and accessible location for research activities. CONCLUSIONS: There are a small number of key differences in the development processes for patient decision aids in which members of vulnerable populations were or were not specifically involved. Some of these practices may require additional time or resources. To address health inequities, researchers, communities and funders may need to increase awareness of these approaches and plan accordingly.

The Health Impacts of Climate Change: A Continuing Medical Education Needs Assessment Framework.

INTRODUCTION: As the health consequences of climate change (CC) will likely become more manifest in the future, family physicians have to be knowledgeable about these impacts and the ways in which they can affect their patients. The main aim of this study was to propose a competency framework and questionnaire used to conduct a needs analysis to identify and prioritize family physicians' real educational needs regarding the health impacts of CC. METHODS: A mixed method combining a qualitative interview and a quantitative online questionnaire was used (n = 24 physicians). The interview assessed key beliefs related to participating in an online continuing medical education (eCME) activity on the health impacts of climate change, and the perception of the key factors or conditions required to ensure the family physicians' satisfaction with this eCME activity. The questionnaire assessed the current and desired levels of competency on five general training themes: general knowledge about CC; heat-related illnesses; CC, extreme weather events and modification of vector-borne and zoonotic diseases; CC, extreme weather events and modification of water-borne diseases; and mental health impacts of natural disasters. RESULTS: Results revealed the need for improved medical education on climate change and health. Results also add to the literature by showing that a 3-hour eCME activity covering these topics would be useful and would allow family physicians to use this knowledge in their daily practice, notably through prevention and counseling. DISCUSSION: Introducing a CME needs assessment framework and a generic instrument that reflects family physicians' needs regarding the health impacts of CC has the added advantage of standardizing the assessment procedure.

User-centered design and the development of patient decision aids: protocol for a systematic review.

BACKGROUND: Providing patient-centered care requires that patients partner in their personal health-care decisions to the full extent desired. Patient decision aids facilitate processes of shared decision-making between patients and their clinicians by presenting relevant scientific information in balanced, understandable ways, helping clarify patients' goals, and guiding decision-making processes. Although international standards stipulate that patients and clinicians should be involved in decision aid development, little is known about how such involvement currently occurs, let alone best practices. This systematic review consisting of three interlinked subreviews seeks to describe current practices of user involvement in the development of patient decision aids, compare these to practices of user-centered design, and identify promising strategies. METHODS/DESIGN: A research team that includes patient and clinician representatives, decision aid developers, and systematic review method experts will guide this review according to the Cochrane Handbook and PRISMA reporting guidelines. A medical librarian will hand search key references and use a peer-reviewed search strategy to search MEDLINE, EMBASE, PubMed, Web of Science, the Cochrane Library, the ACM library, IEEE Xplore, and Google Scholar. We will identify articles across all languages and years describing the development or evaluation of a patient decision aid, or the application of user-centered design or human-centered design to tools intended for patient use. Two independent reviewers will assess article eligibility and extract data into a matrix using a structured pilot-tested form based on a conceptual framework of user-centered design. We will synthesize evidence to describe how research teams have included users in their development process and compare these practices to user-centered design methods. If data permit, we will develop a measure of the user-centeredness of development processes and identify practices that are likely to be optimal. DISCUSSION: This systematic review will provide evidence of current practices to inform approaches for involving patients and other stakeholders in the development of patient decision aids. We anticipate that the results will help move towards the establishment of best practices for the development of patient-centered tools and, in turn, help improve the experiences of people who face difficult health decisions. SYSTEMATIC REVIEW REGISTRATION: PROSPERO CRD42014013241.